Eye tracking system with holographic film decoder

ABSTRACT

A volume holographic film (such as a photopolymer) that is pre-recorded with patterns subsequently is used to encode LED or low-power laser light reflections from an eye into a binary pattern that can be read at very high speeds by a relatively simple complementary metal-oxide-semiconductor (CMOS) sensor that may be similar to a high framerate, low resolution mouse sensor. The low-resolution mono images from the film are translated into eye poses using, for instance, a look up table that correlates binary patterns to X, Y positions or using a pre-trained convolutional neural network to robustly interpret many variations of the binary patterns for conversion to X, Y positions.

FIELD

The application relates generally to eye tracking systems withholographic film decoders.

BACKGROUND

Many applications use eye tracking for useful purposes, such aspresenting certain views or user interfaces based on the direction aperson is looking. As but one example of the use of eye tracking ispresented in the present assignee's USPP 2018/0096518, titled “FIELD OFVIEW (FOV) THROTTLING OF VIRTUAL REALITY (VR) CONTENT IN A HEAD MOUNTEDDISPLAY”.

As understood herein, when executing eye tracking multipleconsiderations can compete with each other. For example, it is desirableboth to use little power and relatively low computational resourceswhile still providing very high speed and accurate eye tracking.

SUMMARY

Accordingly, in some example embodiments a volume holographic material(such as a photopolymer film) is used to encode LED or ultra low-powerlaser light reflections from an eye into a binary pattern that can beread at very high speeds by a relatively simple complementarymetal-oxide-semiconductor (CMOS) sensor that may be similar to a highframerate, low resolution mouse sensor.

In example implementations, a pre-recorded holographic film is used todecode the infrared LED or ultra-low power laser light intopixel-aligned binary patterns onto a very high framerate (>1000 Hz), lowresolution (<64×64 pixels) sensor using a simple processor such as anapplication specific integrated circuit (ASIC) or microcontroller unit(MCU) embedded onto the sensor. The holographic film performs the taskstypically associated with computer vision processes, which is theclassification of an image of the eye to pose values. The ASIC/MCUtranslates the low-resolution mono images into eye pupil poseinformation that is relative to the sensor. The embedded ASIC/MICU canuse either a look up table that correlates binary patterns from thesensor image to pose values, or a pre-trained convolutional neuralnetwork (CNN) classifier, or any other machine learned classifier torobustly interpret many variations of the binary patterns for conversionto pose values. “Pose” refers to the location of the eye and theorientation (direction) in which it is looking, typically as indicatedby the pupil. Eye pose indicates direction of gaze of a user, which canbe input as eye tracking to computer software, e.g., a computer game.

Accordingly, in an example, a method includes directing light from anencoding laser onto a human eye, which reflects the light onto aholographic film to establish coded emissions on respective regions ofthe film. The reflected light from the eye establishes reference lightthat impinges on the holographic film and to produce object light by theprocess of interference of the reference light with particles (typicallysilver halide) in the film, which establishes a binary code imagecorresponding to the eyeball pose.

Subsequently, e.g., during eye tracking for game play, the film isilluminated using at least one reflection of light from a person's eye.The film can be juxtaposed with at least one sensor to sense light fromareas of the film illuminated by the reflection of light from a person'seye and representing at least one of the coded emissions. The methodincludes decoding signals from the sensor representing the at least onecoded emission to return a respective position of the eye.

In some implementations, light from the reflection of light from aperson's eye is infrared (IR). In non-limiting examples, the sensorincludes at least one complementary metal-oxide-semiconductor (CMOS)sensor. In example embodiments, the method may be executed at least inpart by a processor that can be implemented by, e.g., applicationspecific integrated circuit (ASIC) or microcontroller unit (MCU). Theprocessor may be embedded onto/into the sensor.

In another aspect, an apparatus includes at least one light source andat least one holographically recorded film having plural coded regions,with each coded region representing a code different from other codedregions on the film. The apparatus also includes at least one sensor tosense light from at least one coded region of the film illuminated by areflection from an eye of light from the light source. At least onedecoder is configured for decoding signals from the sensor representingthe at least one coded region to return a respective position of theeye.

In another aspect, an apparatus includes at least one holographicallyrecorded film having plural coded regions. Each coded region representsa code different from other coded regions on the film. One or more datastorage media and/or computing units correlate the coded regions torespective positions of an eye. In some other non-limitingimplementations, the holographic material could be a thin hologram, asurface relief hologram or other forms of material structure to performlight interference. In addition, reflective holograms can be usedinstead of transmissive holograms. For simplicity, but withoutlimitations, the present application will refer to thick volumetransmission holograms within a photopolymer film containing silverhalide crystals as holographic film.

The details of the present application, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1-8 and 10 depict example methods for pre-recording a holographicfilm, with FIG. 1 being a block diagram of an encoding laser in a firstposition, illuminating a first coded reflector;

FIGS. 2-5 are schematic views of various types of example robust codesthat can be established by the reflectors shown in FIG. 1;

FIG. 6 is a block diagram of the encoding laser in a second position,illuminating a second coded reflector, with the motor and mechanism formoving the laser removed for clarity;

FIG. 7 schematically illustrates a data structure that correlates laserpositions to specific codes;

FIG. 8 is a block diagram of an encoding laser in a reflectiveconfiguration;

FIG. 9 illustrates how the pre-recorded holographic film is used todetect eye position, with a block diagram of an illuminator such as areflection from a person's eye of an IR laser illuminating theholographic film with emissions from the film being detected by asensor;

FIG. 10 is a flow chart of example logic for establishing the robustcodes on the holographic film;

FIG. 11 is a flow chart of example logic for reading the codes on thefilm;

FIG. 12 illustrates that the object bearing the film/substrate assemblycan be a hand-held game controller;

FIG. 12A illustrates that the object bearing the film/substrate assemblycan be a Head Mounted Display;

FIG. 13 illustrates that the object bearing the film/substrate assemblycan be a glasses-like headset, schematically showing the illuminatorwith a light pipe; and

FIG. 14 is a block diagram of an example system in accordance withpresent principles.

DETAILED DESCRIPTION

This disclosure relates generally to computer ecosystems includingaspects of consumer electronics (CE) device networks such as but notlimited to computer game networks. A system herein may include serverand client components, connected over a network such that data may beexchanged between the client and server components. The clientcomponents may include one or more computing devices including gameconsoles such as Sony PlayStation® or a game console made by Microsoftor Nintendo or other manufacturer of virtual reality (VR) headsets,augmented reality (AR) headsets, portable televisions (e.g. smart TVs,Internet-enabled TVs), portable computers such as laptops and tabletcomputers, and other mobile devices including smart phones andadditional examples discussed below. These client devices may operatewith a variety of operating environments. For example, some of theclient computers may employ, as examples, Linux operating systems,operating systems from Microsoft, or a Unix operating system, oroperating systems produced by Apple Computer or Google. These operatingenvironments may be used to execute one or more browsing programs, suchas a browser made by Microsoft or Google or Mozilla or other browserprograms that can access websites hosted by the Internet serversdiscussed below. Also, an operating environment according to presentprinciples may be used to execute one or more computer game programs.

Servers and/or gateways may include one or more processors executinginstructions that configure the servers to receive and transmit dataover a network such as the Internet. Or, a client and server can beconnected over a local intranet or a virtual private network. A serveror controller may be instantiated by a game console such as a SonyPlayStation®, a personal computer, etc.

Information may be exchanged over a network between the clients andservers. To this end and for security, servers and/or clients caninclude firewalls, load balancers, temporary storages, and proxies, andother network infrastructure for reliability and security. One or moreservers may form an apparatus that implement methods of providing asecure community such as an online social website to network members.FIG. 14 described below provides example components that may be usedherein in the appropriate combinations.

As used herein, instructions refer to computer-implemented steps forprocessing information in the system. Instructions can be implemented insoftware, firmware or hardware and include any type of programmed stepundertaken by components of the system.

A processor may be any conventional general-purpose single- ormulti-chip processor that can execute logic by means of various linessuch as address lines, data lines, and control lines and registers andshift registers.

Software modules described by way of the flow charts and user interfacesherein can include various sub-routines, procedures, etc. Withoutlimiting the disclosure, logic stated to be executed by a particularmodule can be redistributed to other software modules and/or combinedtogether in a single module and/or made available in a shareablelibrary.

Present principles described herein can be implemented as hardware,software, firmware, or combinations thereof; hence, illustrativecomponents, blocks, modules, circuits, and steps are set forth in termsof their functionality.

Further to what has been alluded to above, logical blocks, modules, andcircuits described below can be implemented or performed with ageneral-purpose processor, a digital signal processor (DSP), a fieldprogrammable gate array (FPGA) or other programmable logic device suchas an application specific integrated circuit (ASIC), discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. A processorcan be implemented by a controller or state machine or a combination ofcomputing devices.

The functions and methods described below, when implemented in software,can be written in an appropriate language such as but not limited toJava, C# or C++, and can be stored on or transmitted through acomputer-readable storage medium such as a random access memory (RAM),read-only memory (ROM), electrically erasable programmable read-onlymemory (EEPROM), compact disk read-only memory (CD-ROM) or other opticaldisk storage such as digital versatile disc (DVD), magnetic disk storageor other magnetic storage devices including removable thumb drives, etc.A connection may establish a computer-readable medium. Such connectionscan include, as examples, hard-wired cables including fiber optic andcoaxial wires and digital subscriber line (DSL) and twisted pair wires.Such connections may include wireless communication connectionsincluding infrared and radio.

Components included in one embodiment can be used in other embodimentsin any appropriate combination. For example, any of the variouscomponents described herein and/or depicted in the Figures may becombined, interchanged or excluded from other embodiments.

“A system having at least one of A, B, and C” (likewise “a system havingat least one of A, B, or C” and “a system having at least one of A, B,C”) includes systems that have A alone, B alone, C alone, A and Btogether, A and C together, B and C together, and/or A, B, and Ctogether, etc.

FIG. 1 illustrates a system 10 that includes one or more encoding lasers12 that may emit light through one or more adjustable polarizers 14 ontoa reference or calibration human or mechanical eyeball 15, which in turnreflects light onto a selected reflector A in an array 16 of reflectors.The polarizers 14 can be optional. Each eyeball position may deflectlight onto a reflector in the array 16 to establish, when interactingwith reference light described further below, an interference pattern,such as a binary pattern, of light deflection that creates a robust codecorresponding to the pose of the eye that produced the interferencepattern on the particular area of film. It should also be appreciatedthat instead of a mechanical eyeball, a digital simulation of an eyeballcan be utilized and realized through the use of a spatial lightmodulator such as a Liquid Crystal on Silicon display device typicallyreferred to as LCOS. The LCOS spatial light modulator or any otherspatial light modulator can be used to reflect the encoding laser lightto produce a holographic projection of the simulated eyeball, such thatit approximates the reflected light from a real or mechanical eyeball.

As understood herein, the process of recording similar interferencepatterns (due to similar eye poses) between adjoining areas on theholographic film may produce similar binary coded images such that itwould be difficult to determine the correct eye pose. In such a case,altering the polarization of the light during the recording process canincrease the signal uniqueness of the light from each similar eye poseto each neighboring recording area on the holographic film. Whenpolarizers are used, if the laser/LED light source's polarization stateis sequentially changed and the sensor has sufficient frame rate,multiple alternating polarization state frames can be used beforedetermining the actual eyeball pose. For example, if the sensor frameper second (FPS) is 240 hz, an “S” Polarization state can be used forframe 0 and a “P” Polarization state can be used for frame 1. At the endof frame 1, the results of the binary patterns from frame 0 and frame 1are compared and the frame with the largest correlation (based on sensorpixel intensities) of binary patterns to prerecorded values (forexample, via lookup table or CNN classifier) is used for a result at aframe rate of 120 hz.

It should be noted that the use of altering light polarization toimprove binary image decoding is just one example, other methods couldinclude altering the wavelength of the light or by using light sourcesfrom differing positions and orientations.

In the example shown the eyeball positions are shown reflecting onto atwo-dimensional array of mirrors, but they could be reflected onto athree-dimensional array of transparent objects as shown by the x-y-zaxes 18. Alternatively, other forms of light reflectors or lightdiffracting objects can be used, including but not limited to a LCOSspatial light modulator. Reference light 20 from the encoding laser 12that does not impinge on a reflector can interfere with object light 22from the calibration eyeball, with the resulting interference patternbeing encoded in a region 24 of a holographic film 26. Once illuminationof a first eyeball pose “A” is encoded onto the region 24 of the film26, the eyeball changes pose and the holographic film 26 is moved toexpose a different region under the aperture mask 34 to illuminateanother one of the areas of the film, establishing its own unique code.

Prior to further explanation of present techniques, reference isdirected to FIGS. 2-5, which illustrate various non-limiting examples ofrobust unique codes that can be established and encoded in its ownrespective region of the film 26. FIG. 2 shows a single “splotch” 200with a unique configuration. Each eyeball pose may encode its ownrespective splotch. FIG. 3 shows a series of unique linear codes, e.g.,first and second codes 300, 302 that can be established by respectiveeyeball poses. FIG. 4 illustrates that each unique code may be a quickresponse (QR) code 400, while FIG. 5 shows that each unique code may bea bar code 500. Combinations of the codes in FIGS. 2-5 may be used.

The location (also referred to herein as “position”) of the encodinglaser 12 is in with respect to the film 26 when irradiating thereference eyeball to encode the interference pattern in the region 24 byrecording onto the film, the interference pattern being formed from thereflection of the laser light due to the pose of the eyeball (theposition of the eyeball and the direction in which the pupil is directedrelative to the center of the eye) and the reflection off the objectsthat create the unique code.

As shown in FIG. 6, once the region 24 has encoded the unique fringepattern from the eyeball orientation A, the reference, calibration,mechanical or digitally simulated eyeball 15 is moved, e.g., rotated,such that the resulting interference pattern from the direct beam 600and deflected beam 602 is encoded in a second region 604 that issignificantly distanced from the first region 24. As mentioned for FIG.1, the region 604 under light exposure from the beams 600 and 602, maybe due to the holographic film 26 being moved. In the example shown,while the first and second regions encode respective unique codes thatare associated with respective eyeball poses that are only a singleincrement of location recording apart, they may be physically separatedfrom each other by regions of the film 26 that encode other codesassociated with other eyeball poses. Also, successive eyeball poses maybe irradiated with respective different polarizations. In this way, whenthe film 26 subsequently is used to determine the pose of a differenthuman eyeball during, e.g., game play or for other eye trackingapplications as described more fully below, discrimination of theprecise location of the eyeball is made more robust by reducing thepossibility of cross-talk caused by similar poses of the eyeball lightreflections illuminating similar encodings on the holographic film.

FIG. 7 shows an example data structure that may be recorded on, e.g.,disk-based or solid-state memory in which laser locations in a firstcolumn 700 are correlated with respective robust codes in a column 702,for purposes to be shortly disclosed.

While FIGS. 1 and 6 show a transmissive system, FIG. 8 shows that alaser 800 may be used to sequentially irradiate each of a series ofeyeball poses 802 (schematically shown as rectangles) in a reflectivearrangement to encode the respective codes onto the film 26.

It may now be appreciated that once the film 26 has been encoded asdescribed above, when another light beam such as a reflection ofinfrared (IR) light such as from a lower power laser or light emittingdiode (LED) subsequently illuminates a human eye and is reflected ontothe film, the light beam will illuminate the region of film that wasencoded by the encoding laser 12 when the reference eyeball was in thesame relative orientation/position to the film 26 as the subsequenteyeball is in. The light beam in turn represents the location of theperson's eye, as IR light predominantly will reflect from the pupil.

FIG. 9 illustrates an indicator light source 900 (with adjustablepolarizer not shown) illuminating a human eye 901, with the reflectionfrom the eye in turn illuminating one of plural encoded regions 902 onthe film 26, with each region 902 encoding a unique robust code that iscorrelated to a respective pose of the reference eye as described above.The light source 900 may be, e.g., an ultra low power IR laser or lightemitting diode. A sensor 904 such as but not limited to a charge-coupleddevice (CCD), complementary metal-oxide semiconductor (CMOS) detector,or photodiode array detector senses light emitted from the film 26 (and,hence, the unique code of the region 902 that is illuminated) and sendsa signal representative thereof to one or more processors 906, such asan application specific integrated circuit (ASIC) or microcontrollerunit (MCU) embedded onto/into the sensor 904 to render a combinedfilm/sensor assembly 907 with, in some implementations, an embeddedprocessor 906. The sensor 904 may be a very high framerate (>1000 Hz),low resolution (<64×64 pixels) sensor.

Note that the entire eyeball can reflect and interfere with the lowpower illumination light (e.g., infrared laser light), bouncing back tothe holographic film 26. The film 26, if desired, can be segmented intothe regions 902 to match the sensor 904 pixels on a one-to-one basiswith a region 902 being overlaid onto a camera pixel. Each region 902can contain the encoded reference interference pattern(s) that produce astrong correlation to a reference position/angle of the eye.Constructive or destructive interference can be used to provide thecorrelation along the lines of the principles of HolographicInterferometry.

The processor 906 can execute image recognition to determine whichunique code is received and access the data structure shown in FIG. 7 tocorrelate the code to a pose of the eye 901 with respect to the film 26,such as an X-Y location in Cartesian coordinates. The processor 906 mayexecute a software-based computer game 908 and output demanded imagesfrom the game 908 onto a display 910, with game execution (and, hence,the demanded images) using, if desired, the eye pose to alter the gameimages. This is amplified on further below.

The indicator light source 900 may be an infrared (IR) laser. In someembodiments the wavelength of the light emitted by the indicator lightsource 900 may be greater than 1,000 nanometers, e.g., 1,440 nm toensure that a game player does not see the laser light. The laser may bepulsed using a pulse repetition rate (PRR) that uniquely identifies thelaser from other nearby indicator lasers. The laser may be modulated ata very high carrier frequency, e.g., in excess of one megahertz toincrease the uniqueness of the light compared to other light sourceslike sunlight.

If desired, in some example implementations an encoding holographic film912 may be placed between the light source 900 and film 26 (in theexample shown, in the light path from the light source 900 to the eye901) to provide a more consistent and defined interference pattern onthe decoding film 26/sensor 904. By identifying the various lightintensities that fall on the sensor 904, the position/angle of the eyecan be determined.

FIG. 10 shows the logic of the encoding technique described above whileFIG. 11 shows the logic of the subsequent eye location determinationtechnique, with some or all of the logic steps being controlled by anyof the processors described herein. Commencing at block 1000, thereference eyeball is rotated/moved to the first pose relative to thefilm 26 and activated to illuminate a first reflector A at block 1002.The code is captured or encoded at block 1004 on the holographic film 26in a first region A of the film (region 24 in FIGS. 1 and 6). This isachieved by the holographic recording process (for example, exposure ofa photopolymer film to laser light).

Proceeding to block 1006 the reference eyeball is rotated to the nextpose relative to the film 12, and if desired its polarization is changedat block 1008 for reasons explained above. A second reflector B isilluminated by the laser at block 1010 and its code captured (encoded)in the film 26 at block 1012. The described process of rotating thereference eyeball, changing polarization if desired, and successivelyilluminating reflectors continues at block 1014 for subsequent locations3, . . . N to encode subsequent respective unique reflector codes C, . .. N onto the film 26, with each code being recorded and correlated tothe respective location information of the reference eyeball at block1016.

In examples, a silver halide crystal photo polymer holographic film isused.

Note that a mechanical eye of known properties may be used as thereference eyeball. Note that an encoding pattern such as any shown inFIGS. 2-5 and described above may be selected that is easily detectableand that increases the signal to noise ratio of the decoding process.The pattern may be based on the geometry of the eye. Preferably, theholographic film effectively compensates so that only light reflectedoff a pupil is clearly defined. An anti-distortion pattern may be used,and the pattern may be designed so that the “bump” of the iris is morein focus. The film may be exposed multiple times to increase the numberof encodings that be recorded.

Recalling the subsequent eye location determination system of FIG. 9 andturning now to related FIG. 11, at block 1100, if desired the eye oreyes of a person as illuminated by the indicator light source 900 iscalibrated to a location relative to the film 26 to approximate that ofthe reference eyeball in the earlier encoding process. This may be done,e.g., by instructing a user to mount the indicator light source at acertain point or location, e.g., on the top middle of a display on whicha computer game is to be displayed and look in a specified direction,e.g., straight ahead. This calibration location may be supplied to acomputer game.

Proceeding to block 1102, the film 26 is illuminated with lightreflected from the eye from the indicator light source 900. The sensor904 senses the resultant unique robust code pattern of light emittedfrom the film and its signal representative thereof is received at block1104. Image recognition is applied to the signal to recognize the codeat block 1106, which is then used at block 1108 as entering argument to,e.g., the data structure of FIG. 7 to return the corresponding pose ofthe eye with respect to the film, such as the X-Y location of the pupilof the eye relative to the location of surrounding eye tissue. This posemay be output at block 1110 to an AR or VR computer game console orother electronic device that uses eye tracking for various purposes. Insome embodiments, instead of or in addition to a lookup table, theprocessor (e.g., an embedded ASIC/MCU) can use a pre-trainedconvolutional neural network classifier to robustly interpret manyvariations of the binary patterns for conversion to X, Y positions.

In example embodiments, each holographic film may be established byplural sub-films.

Various eye shapes and distances between pupils may be adapted for, withvariations on the shape of the eye being be encoded onto the film. Aftercalibration, a person whose eyes are to be tracked may elect to wearcontact lenses, which may distort the current system, so the eyes may beimaged with contacts and without to calibrate for contact wearing. Beamsteering may be affected by changing an angle of a mirror reflectinglight from the eye to the sensor based on whether the person hascontacts, how hard the contacts are, etc. To account for jostling duringimaging, a tilting mirror can be adjusted until a highest signal isdetected. Other forms adjustments can be used to ensure the lightreflected from the eye falls onto the sensor to ensure accurate eyetracking via the present application.

A movable film/sensor assembly 907 may be implemented by a VR or ARheadset such as the ones shown in FIGS. 12A and 14 and described furtherbelow. A single headset may include multiple assemblies that can beilluminated by reflections from respective eyes. Since the arrangementof plural film/sensor assemblies on a headset is known, their relativelocations with respect to each other also are known.

Or, the movable film/sensor assembly 907 may be implemented by a gamecontroller such as the controller 1200 shown in FIG. 12 or ahead-mounted display 1200A as shown in FIG. 12A. In FIG. 12A, the HMD1200A comprises lenses 1210 and at least one holographic decoding filmand CMOS sensor assembly 1220 consistent with principles above. At leastone laser light source (and optionally holographic encoding film)assembly 1222 is positioned to illuminate the eye, reflections of whichimpinge upon the holographic decoding film and CMOS sensor assembly1220.

Yet again, the movable film/sensor assembly 907 may be implemented by aneyeglasses-type frame 1300 (FIG. 13). A laser or other light source(e.g., IR LED) 1302 may be mounted in the frame and a light pipe 1304may be used to direct laser light onto glasses-type displays 1306. Yetagain, the film/CMOS assembly 907 may be implemented in a wristwatch,mobile phone, tablet, laptop or any of the example devices such as thedevice shown in, e.g., FIG. 14.

Each movable film/sensor assembly 907 can determine one or more eyelocations as described above and wirelessly report the location to thegame processor. Or, the assembly 907 can simply send a signalrepresenting the unique code being illuminated to the game processor forderivation of the location by the game processor. Regardless, the gameprocessor may then know, for example, the location of a player's eyesand tailor presentation accordingly.

Now referring to FIG. 14, an example system 1400 is shown, which mayinclude one or more of the example devices mentioned below in accordancewith present principles. The first of the example devices included inthe system 1410 is a consumer electronics (CE) device such as an audiovideo device (AVD) 1412 such as but not limited to an Internet-enabledTV with a TV tuner (equivalently, set top box controlling a TV).However, the AVD 1412 alternatively may be an appliance or householditem, e.g. computerized Internet enabled refrigerator, washer, or dryer.The AVD 1412 alternatively may also be a computerized Internet enabled(“smart”) telephone, a tablet computer, a notebook computer, a wearablecomputerized device such as e.g. computerized Internet-enabled watch, acomputerized Internet-enabled bracelet, other computerizedInternet-enabled devices, a computerized Internet-enabled music player,computerized Internet-enabled head phones, a computerizedInternet-enabled implantable device such as an implantable skin device,etc. Regardless, it is to be understood that the AVD 1412 is configuredto undertake present principles (e.g. communicate with other CE devicesto undertake present principles, execute the logic described herein, andperform any other functions and/or operations described herein).

Accordingly, to undertake such principles the AVD 1412 can beestablished by some or all of the components shown in FIG. 14. Forexample, the AVD 1412 can include one or more displays 1414 that may beimplemented by a high definition or ultra-high definition “4K” or higherflat screen and that may be touch-enabled for receiving user inputsignals via touches on the display. The AVD 1412 may include one or morespeakers 1416 for outputting audio in accordance with presentprinciples, and at least one additional input device 1418 such as e.g.an audio receiver/microphone for e.g. entering audible commands to theAVD 1412 to control the AVD 1412. The example AVD 1412 may also includeone or more network interfaces 1420 for communication over at least onenetwork 1422 such as the Internet, an WAN, an LAN, etc. under control ofone or more processors 1424 including. A graphics processor 1424A mayalso be included. Thus, the interface 1420 may be, without limitation, aWi-Fi transceiver, which is an example of a wireless computer networkinterface, such as but not limited to a mesh network transceiver. It isto be understood that the processor 1424 controls the AVD 1412 toundertake present principles, including the other elements of the AVD1412 described herein such as e.g. controlling the display 1414 topresent images thereon and receiving input therefrom. Furthermore, notethe network interface 1420 may be, e.g., a wired or wireless modem orrouter, or other appropriate interface such as, e.g., a wirelesstelephony transceiver, or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the AVD 1412 may also include one or moreinput ports 1426 such as, e.g., a high definition multimedia interface(HDMI) port or a USB port to physically connect (e.g. using a wiredconnection) to another CE device and/or a headphone port to connectheadphones to the AVD 1412 for presentation of audio from the AVD 1412to a user through the headphones. For example, the input port 1426 maybe connected via wire or wirelessly to a cable or satellite source 1426a of audio video content. Thus, the source 1426 a may be, e.g., aseparate or integrated set top box, or a satellite receiver. Or, thesource 1426 a may be a game console or disk player containing contentthat might be regarded by a user as a favorite for channel assignationpurposes described further below. The source 1426 a when implemented asa game console may include some or all of the components described belowin relation to the CE device 1444.

The AVD 1412 may further include one or more computer memories 1428 suchas disk-based or solid-state storage that are not transitory signals, insome cases embodied in the chassis of the AVD as standalone devices oras a personal video recording device (PVR) or video disk player eitherinternal or external to the chassis of the AVD for playing back AVprograms or as removable memory media. Also, in some embodiments, theAVD 1412 can include a position or location receiver such as but notlimited to a cellphone receiver, GPS receiver and/or altimeter 1430 thatis configured to e.g. receive geographic position information from atleast one satellite or cellphone tower and provide the information tothe processor 1424 and/or determine an altitude at which the AVD 1412 isdisposed in conjunction with the processor 1424. However, it is to beunderstood that that another suitable position receiver other than acellphone receiver, GPS receiver and/or altimeter may be used inaccordance with present principles to e.g. determine the location of theAVD 1412 in e.g. all three dimensions.

Continuing the description of the AVD 1412, in some embodiments the AVD1412 may include one or more cameras 2632 that may be, e.g., a thermalimaging camera, a digital camera such as a webcam, and/or a cameraintegrated into the AVD 1412 and controllable by the processor 1424 togather pictures/images and/or video in accordance with presentprinciples. Also included on the AVD 1412 may be a Bluetooth transceiver1434 and other Near Field Communication (NFC) element 1436 forcommunication with other devices using Bluetooth and/or NFC technology,respectively. An example NFC element can be a radio frequencyidentification (RFID) element.

Further still, the AVD 1412 may include one or more auxiliary sensors1437 (e.g., a motion sensor such as an accelerometer, gyroscope,cyclometer, or a magnetic sensor, an infrared (IR) sensor, an opticalsensor, a speed and/or cadence sensor, a gesture sensor (e.g. forsensing gesture command), etc.) providing input to the processor 1424.The AVD 1412 may include an over-the-air TV broadcast port 1438 forreceiving OTA TV broadcasts providing input to the processor 1424. Inaddition to the foregoing, it is noted that the AVD 1412 may alsoinclude an infrared (IR) transmitter and/or IR receiver and/or IRtransceiver 1442 such as an IR data association (IRDA) device. A battery(not shown) may be provided for powering the AVD 1412.

Still referring to FIG. 14, in addition to the AVD 1412, the system 1400may include one or more other CE device types. In one example, a firstCE device 1444 may be used to send computer game audio and video to theAVD 1412 via commands sent directly to the AVD 1412 and/or through thebelow-described server while a second CE device 1446 may include similarcomponents as the first CE device 1444. In the example shown, the secondCE device 1446 may be configured as a VR headset worn by a player 1447as shown. In the example shown, only two CE devices 1444, 1446 areshown, it being understood that fewer or greater devices may be used.For example, principles below discuss multiple players 1447 withrespective headsets communicating with each other during play of acomputer game sourced by a game console to one or more AVD 1412, as anexample of a multiuser voice chat system.

In the example shown, to illustrate present principles all three devices1412, 1444, 1446 are assumed to be members of an entertainment networkin, e.g., a home, or at least to be present in proximity to each otherin a location such as a house. However, present principles are notlimited to a particular location, illustrated by dashed lines 1448,unless explicitly claimed otherwise. Any or all of the devices in FIG.14 can implement any one or more of the lasers, films, and sensorsdescribed previously.

The example non-limiting first CE device 1444 may be established by anyone of the above-mentioned devices, for example, a portable wirelesslaptop computer or notebook computer or game controller (also referredto as “console”), and accordingly may have one or more of the componentsdescribed below. The first CE device 1444 may be a remote control (RC)for, e.g., issuing AV play and pause commands to the AVD 1412, or it maybe a more sophisticated device such as a tablet computer, a gamecontroller communicating via wired or wireless link with the AVD 1412, apersonal computer, a wireless telephone, etc.

Accordingly, the first CE device 1444 may include one or more displays1450 that may be touch-enabled for receiving user input signals viatouches on the display. The first CE device 1444 may include one or morespeakers 1452 for outputting audio in accordance with presentprinciples, and at least one additional input device 1454 such as e.g.an audio receiver/microphone for e.g. entering audible commands to thefirst CE device 1444 to control the device 1444. The example first CEdevice 1444 may also include one or more network interfaces 1456 forcommunication over the network 1422 under control of one or more CEdevice processors 1458. A graphics processor 1458A may also be included.Thus, the interface 1456 may be, without limitation, a Wi-Fitransceiver, which is an example of a wireless computer networkinterface, including mesh network interfaces. It is to be understoodthat the processor 1458 controls the first CE device 1444 to undertakepresent principles, including the other elements of the first CE device1444 described herein such as e.g. controlling the display 1450 topresent images thereon and receiving input therefrom. Furthermore, notethe network interface 1456 may be, e.g., a wired or wireless modem orrouter, or other appropriate interface such as, e.g., a wirelesstelephony transceiver, or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the first CE device 1444 may also includeone or more input ports 1460 such as, e.g., a HDMI port or a USB port tophysically connect (e.g. using a wired connection) to another CE deviceand/or a headphone port to connect headphones to the first CE device1444 for presentation of audio from the first CE device 1444 to a userthrough the headphones. The first CE device 1444 may further include oneor more tangible computer readable storage medium 1462 such asdisk-based or solid-state storage. Also in some embodiments, the firstCE device 1444 can include a position or location receiver such as butnot limited to a cellphone and/or GPS receiver and/or altimeter 1464that is configured to e.g. receive geographic position information fromat least one satellite and/or cell tower, using triangulation, andprovide the information to the CE device processor 1458 and/or determinean altitude at which the first CE device 1444 is disposed in conjunctionwith the CE device processor 1458. However, it is to be understood thatthat another suitable position receiver other than a cellphone and/orGPS receiver and/or altimeter may be used in accordance with presentprinciples to e.g. determine the location of the first CE device 1444 ine.g. all three dimensions.

Continuing the description of the first CE device 1444, in someembodiments the first CE device 1444 may include one or more cameras1466 that may be, e.g., a thermal imaging camera, a digital camera suchas a webcam, and/or a camera integrated into the first CE device 1444and controllable by the CE device processor 1458 to gatherpictures/images and/or video in accordance with present principles. Alsoincluded on the first CE device 1444 may be a Bluetooth transceiver 1468and other Near Field Communication (NFC) element 1470 for communicationwith other devices using Bluetooth and/or NFC technology, respectively.An example NFC element can be a radio frequency identification (RFID)element.

Further still, the first CE device 1444 may include one or moreauxiliary sensors 1472 (e.g., a motion sensor such as an accelerometer,gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor, anoptical sensor, a speed and/or cadence sensor, a gesture sensor (e.g.for sensing gesture command), etc.) providing input to the CE deviceprocessor 1458. The first CE device 1444 may include still other sensorssuch as e.g. one or more climate sensors 1474 (e.g. barometers, humiditysensors, wind sensors, light sensors, temperature sensors, etc.) and/orone or more biometric sensors 1476 providing input to the CE deviceprocessor 1458. In addition to the foregoing, it is noted that in someembodiments the first CE device 1444 may also include an infrared (IR)transmitter and/or IR receiver and/or IR transceiver 1478 such as an IRdata association (IRDA) device. A battery (not shown) may be providedfor powering the first CE device 1444. The CE device 1444 maycommunicate with the AVD 1412 through any of the above-describedcommunication modes and related components.

The second CE device 1446 may include some or all of the componentsshown for the CE device 1444. Either one or both CE devices may bepowered by one or more batteries.

Now in reference to the afore-mentioned at least one server 1480, itincludes at least one server processor 1482, at least one tangiblecomputer readable storage medium 1484 such as disk-based or solid statestorage, and at least one network interface 1486 that, under control ofthe server processor 1482, allows for communication with the otherdevices of FIG. 14 over the network 1422, and indeed may facilitatecommunication between servers and client devices in accordance withpresent principles. Note that the network interface 1486 may be, e.g., awired or wireless modem or router, Wi-Fi transceiver, or otherappropriate interface such as, e.g., a wireless telephony transceiver.

Accordingly, in some embodiments the server 1480 may be an Internetserver or an entire server “farm”, and may include and perform “cloud”functions such that the devices of the system 1400 may access a “cloud”environment via the server 1480 in example embodiments for, e.g.,network gaming applications. Or, the server 1480 may be implemented byone or more game consoles or other computers in the same room as theother devices shown in FIG. 14 or nearby.

The methods herein may be implemented as software instructions executedby a processor, suitably configured application specific integratedcircuits (ASIC) or field programmable gate array (FPGA) modules, or anyother convenient manner as would be appreciated by those skilled inthose art. Where employed, the software instructions may be embodied ina non-transitory device such as a CD ROM or Flash drive. The softwarecode instructions may alternatively be embodied in a transitoryarrangement such as a radio or optical signal, or via a download overthe internet.

It will be appreciated that whilst present principals have beendescribed with reference to some example embodiments, these are notintended to be limiting, and that various alternative arrangements maybe used to implement the subject matter claimed herein.

What is claimed is:
 1. A method comprising: receiving reflections from ahuman eye of light from an encoding laser on a holographic film toestablish coded emissions on respective regions of the film; correlatingthe coded emissions to respective poses of the human eye; illuminatingthe film using at least one reflection of light from a person's eye, thefilm being juxtaposed with at least one sensor to sense light from areasof the film illuminated by the reflection of light from a person's eyeand representing at least one of the coded emissions; and decodingsignals from the sensor representing the at least one coded emission toreturn a respective pose of the eye.
 2. The method of claim 1, whereinlight from the reflection of light from a person's eye is infrared (IR).3. The method of claim 2, wherein the sensor comprises at least onecomplementary metal-oxide-semiconductor (CMOS) sensor.
 4. The method ofclaim 1, wherein the method is executed at least in part by a processor.5. The method of claim 4, wherein the processor comprises at least oneapplication specific integrated circuit (ASIC).
 6. The method of claim4, wherein the processor comprises at least one microcontroller unit(MCU).
 7. The method of claim 4, comprising embedding the processoronto/into the sensor.
 8. An apparatus, comprising: at least one lightsource; at least one holographically recorded film having plural codedregions, each coded region representing a code different from othercoded regions on the film; at least one sensor to sense light from atleast one coded region of the film illuminated by a reflection from aneye of light from the light source; and at least one decoder configuredfor decoding signals from the sensor representing the at least one codedregion to return a respective pose of the eye.
 9. The apparatus of claim8, wherein the codes comprise respective plural different splotches. 10.The apparatus of claim 8, wherein the codes comprise respective pluraldifferent lines.
 11. The apparatus of claim 8, wherein the codescomprise respective plural different bar codes.
 12. The apparatus ofclaim 8, wherein the codes comprise respective plural different quickresponse (QR) codes.
 13. The apparatus of claim 8, wherein the lightsource comprises at least one light emitting diode (LED).
 14. Theapparatus of claim 8, wherein the sensor comprises at least onecomplementary metal-oxide-semiconductor (CMOS) sensor.
 15. The apparatusof claim 8, wherein the decoder comprises at least one applicationspecific integrated circuit (ASIC).
 16. The apparatus of claim 8,wherein the decoder comprises at least one microcontroller unit (MCU).17. The apparatus of claim 8, wherein the decoder is embedded onto/intothe sensor.
 18. An apparatus, comprising: at least one holographicallyrecorded film having plural coded regions, each coded regionrepresenting a code different from other coded regions on the film; andat least one data storage medium correlating the coded regions torespective poses of an eye.
 19. The apparatus of claim 18, comprising:at least one light source; at least one sensor to sense light emitted bythe light source and reflected from a human eye onto at least one codedregion of the film; and at least one decoder configured for correlatingsignals from the sensor representing the at least one coded region toreturn a respective pose of the eye.
 20. The apparatus of claim 19,wherein the light from the light source is infrared (IR).